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DETAILED ACTION 



1 . This action is responsive to communication: amendment filed on 10/23/03 to the 
application filed on 05/30/97. 

2. Claims 12, 17-30, 51-62, 67-99 are canceled. 

3. Claim 100 is added. 

4. Claims 1-11, 13-16, 31-50, 63-66, 100 are pending in the case. Claims 1. 10 and 
100 are independent claims. 

5. The rejections of claims 1-4, 9-11, 13-16, 63-66 under 35 U.S.C. 103(a) as being 
unpatentable over Shaw in view of Gertler have been withdrawn in view of the 
amendment. 

6. The rejections of claims 5-6 under 35 U.S.C. 103(a) as being unpatentable over 
Shaw in view of Gertler and further in view of Johnson have been withdrawn in view of 
the amendment. 

7. The rejections of claims 7-8 under 35 U.S.C. 103(a) as being unpatentable over 
Shaw in view of Gertler and further in view of Ando have been withdrawn in view of the 
amendment. 

8. The rejections of claims 31-50 under 35 U.S.C. 103(a) as being unpatentable 
over Shaw in view of Gertler and further in view of Caire have been withdrawn in view of 
the amendment. 
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Claim Rejections - 35 USC § 103 



9. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

10. Claims 1-3. 9-11, 14, 63-66 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Robotham et al. (US Pat No. 6,084,590, 7/4/00, filed 10/10/97, 
priority 4/7/97). 

Regarding independent claim 1, Robotham discloses: 

- receiving a stream including a file that integrates media content with 
choreography information within each of at least two objects of the file, each of 
the objects including media content data and choreography information 
associated therewith, the choreography information comprising data indicating an 
author-designated relationship between the objects of the file that defines a 
temporal order of presentation between the objects (col 10, lines 1 1-37: different 
types of media elements are choreographed in space and time within a single 
integrated choreography specification framework) 

- before all objects of the file are received, beginning to render media content 
encapsulated within the file based on the choreography information associated 
with objects received so as to enable an ordered display of the objects received 
based on the choreography information (col 22, line 49 to col 23, line 6: rendering 
the objects in the media file based on the choreography information associated 
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with the media objects to a user; since the display of media segments occurs in a 
choreography model pixel-by-pixel basis during the finish rendering process, the 
render of media content is received as choreographed before all the objects of 
the file are received) 

Robotham does not explicitly disclose the rendering process is unaffected by an input of 
the recipient and the ordered display being independent of a bandwidth of a 
communication channel used to send the multimedia document. 
However, it would have been obvious to one of ordinary skill in the art at the time of the 
invention was made to have modified Robotham to include the unaffected ness by an 
input recipient and the independence of the ordered display of a bandwidth of the 
communication channel used to send the multimedia document for the following reason. 
The fact that the choreography information is provided for generation of the image 
stream of choreographed objects within the virtual stage suggests that the image 
stream be rendered to the recipient as generated with choreography specification. The 
rendering, therefore, is unaffected by any input from recipient. Further, the ordered 
display, since specified in the choreography information in advance, will be rendered in 
the defined order independently of the bandwidth of a communication channel used to 
send the multimedia document. 

Regarding claims 2 and 3, which is dependent on claim 1 , Robotham discloses: 
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- changing one object in the data file (col 12, line 64 to col 13, line 6: changing can 
be made on media objects during choreography process; col 9, lines 10-63: 
altering characteristics of objects) 

- adding an object to the data file (col 10, lines 3-10: the fact that media elements 
can be edited in the way that they can be integrated with one another spatially 
and acoustically suggests adding an object to the data file) 

Regarding claim 9, which is dependent on claim 1, Robotham discloses creating an 
object in the file (col 7, lines 17-32) and locating player data within an object defining a 
player that plays the object (col 10, lines 1 1-37: different types of media elements such 
as video and audio can be treated as if they are produced simultaneously within the 
unified virtual stage suggests defining a player for playing the media elements) 
Robotham does not disclose the created object is an unknown object. However, it 
would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have modified Robotham to include the created object being an unknown 
object for the following reason. Robotham provides the interface for each multimedia 
object thus no matter the object is known or unknown, the system always locates the 
player associated with the multimedia object. 

Independent claim 10 is for a computer system of the method claim 1 , and is rejected 
under the same rationale. 
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Regarding claim 11, which is dependent on claim 10, Robotham discloses that at least 
one object comprises one of a textual file format, an image file format, and a sound file 
format (col 12, lines 6-21 and col 22, lines 49-67: the media object comprises image 
format and audio format). 

Regarding claim 14, which is dependent on claim 10, Robotham discloses that each 
object is a generic element of the hierarchical data file structure, such that any 
combination of objects can be grouped together to form a part of the multimedia 
document (col 2. line 66 to col 3, line 9; col 22, lines 22-29: the choreography 
specification controls the hierarchical time line that defines the appearance of the media 
elements of a scene). 

Regarding claims 63 and 65, which are dependent on claims 1 and 10 respectively, 
Robotham discloses that the ordered display is independent of a recipient software 
program used to render the objects (col 22, lines 13-37, 49 to col 23, line 6: the 
capability of rendering the multimedia stream as choreographed in advance suggests 
that the ordered display be independent of a recipient software program). 

Regarding claims 64 and 66, which are dependent on claims 63 and 65 respectively, as 
mentioned in claims 63 and 65 above, the ordered display is independent of the 
browser used to render the objects so that the display is presented as choreographed 
by the document author. 
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11. Claims 4, 13. 15, 16 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Robotham as applied to claims 1 and 10 above, and further in view of Shaw et al., 
Microsoft Office 6-in-1, Que Corporation 1994, pages 379-380, 384-389, 396-402, 419- 
425, 492-496. 

Regarding claim 4, which is dependent on claim 1, Robotham does not disclose: 

- creating an exclusionary area within the window 

- locating an object within the exclusionary area, the object being selected from a 
group of objects including a framed image, a slide show, framed text, sound data, 
a separator, or a hyperlink 

Shaw discloses: 

- creating an exclusionary area within the window (page 401 , figure 4.4) 

- locating an object within the exclusionary area, the object being selected from a group 
of objects including a framed image, a slide show, framed text, sound data, a separator, 
or a hyperlink (page 401 , figure 4.4: the data in the area within the window can be text 
and graphics). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Shaw into Robotham since Shaw teaches the window 
features for locating objects selected from a group of specific objects providing the 
advantage to include in Robotham to enhance the use of objects from internet instead 
of merely video or audio from the media database. 



Application/Control Number: 08/866,857 Page 8 

Art Unit: 2178 

Regarding clainn 13, which is dependent on claim 10, Robotham does not disclose that 
two or more objects have at least one common attribute, including at least one of a 
command for perception of the object, an ability to pass and receive a message, and an 
ability to supply and retrieve the data embodied in the object. 

Shaw discloses that two or more objects have at least one common attribute, including 
at least one of a command for perception of the object, an ability to pass and receive a 
message, and an ability to supply and retrieve the data embodied in the object (page 
495: since the display of the slides can be set in a temporal order by the document 
author, the slides as in the slide stream has the ability to pass and receive a message to 
automatically advance to the next slide to display the data in the slide). 
It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Shaw into Robotham since Shaw teaches the ability of 
passing and receiving a message between the objects to supply and retrieve data 
embodied in the object providing the advantage of enhancing the relationship among 
the media objects choreographed in the image stream as in Robotham. 

Regarding claims 15 and 16, which are dependent on claim 10, Robotham does not 
disclose the document forms a code segment that receives image information, and 
wherein the image information is used to construct an image frame for a framed image 
that is part of the multimedia document. 
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Shaw discloses that the document forms a code segment that receives image 
information, and wherein the image information is used to construct an image frame for 
a framed image that is part of the multimedia document (pages 400-401). 
It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Shaw into Robotham since Shaw a code segment to 
control the receiving of image information as well as the structure of the media stream 
providing the advantage of utilizing such control to edit the media stream in Robotham 
as desired. 

12. Claims 5-6 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Robotham as applied to claim 1 above, and further in view of Johnson (US Pat No. 
5,892,847, 4/6/99, filed 4/22/96). 

Regarding claims 5 and 6, which are dependent on claims 1 and 5 respectively, 
Robotham does not disclose defining as well as locating the update splash image within 
the data file. 
Johnson discloses: 

- splash image data defining a splash image and locating the splash image data within 
the data file for displaying the splash image on the computer display (col 4, lines 30-50) 

- further updating the splash image to be displayed (col 4, lines 30-63) 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Johnson into Robotham since Johnson teaches the 
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process of displaying of a splash image, which is an element of a multimedia document 
providing the advantage of including a splash image into the image file to enhance the 
image characteristics in Robotham. 

13. Claims 7-8 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Robotham as applied to claim 1 above, and further in viev\^ of Ando (US Pat No. 
5.600,826, 2/4/97). 

Regarding claim 7, which is dependent on claim 1, Robotham does not disclose that 
each object has an address indicating a player that plays the object. 
Ando discloses that each object has an object identifier that stores the position 
information of a data element (col 1 , lines 9-22). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Ando into Robotham since Ando provides the object 
identifier, which is an object address, to quickly find the object in the multimedia 
document in the process of identifying objects and object player. 

Regarding claim 8, which is dependent on claim 1, Robotham does not disclose 
compressing information in each object. 

Ando discloses a data compression/development device can, of course, be incorporated 
into a structured data processor (col 6, lines 38-43). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Ando into Robotham since Ando has the ability of 
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compressing data for high-speed data transmission. This implies there is also an 
information of compressing in each object. The compression feature of Ando provides 
the advantage of faster sending multimedia data over the network channel due to 
reducing the amount of transmitted data. 

14. Claims 31-50 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Robotham as applied to claims 1 and 10 above, and further in view of Cairo et al. (US 
Pat No. 5,663,962, 9/2/97, filed 9/15/95). 

Regarding claim 31, which is dependent on claim 1, and claims 32-34, which are 
dependent on claim 31 , Robotham does not disclose that the choreography information 
further comprises a header, an object archive for storing information about one or more 
objects, the object archive including information about the relationship of the object file 
with the document, and a multiplex section including data for the objects in the 
document. 
Caire discloses: 

- a header (col 1 , lines 65 to col 2, lines 1-2, each packet in the overall stream 
includes a header) 

- an object archive for storing information about the plurality of object files, the 
object archive including information about the level of each object file with the 
hierarchy (col 1 , lines 65 to col 2, lines 1-2, each packet of the multimedia stream 
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stores information; col 1 . lines 37-52, it is desired for instance to insert into the 
complete stream also some subtitles to be displayed during the presentation....) 

- a multiplex section including data for each of the object files of the document (col 
1, lines 65 to col 2, lines 1-9, 45-59) 

- the object files in the multiplex section are each played by a player as the 
multiplex object file is received by a receiver (col 1 , lines 65 to col 2, lines 1-2) 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Cairo into Robotham since Cairo provides the 
choreography and the multiplexing features for a multimedia presentation. The 
combination of Cairo into Robotham would provide more detailed options in the 
relationship of the objects in a multimedia document to effectively controlling and 
changing the presentation of the objects. 

Regarding claims 35, 36-39, which are dependent on claims 31 and 35 respectively, 
Robotham does not disclose an object number counter indicating the number of objects, 
a plurality of object descriptions, each object description describing a corresponding one 
of the objects, and a choreography group providing information about a first group of 
objects, a group object counter indicating the number objects in the choreography 
group, size and type data for each object, header data, data slices of the objects 
interleaved together, and placing one or more slice size data blocks before one or more 
of the interleaved data slices, each slice size data block corresponding to a data slice 
and providing a size of the corresponding data slice. 
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Caire discloses: 

- an object number counter indicating the number of object files (col 2, lines 10-20) 

- a plurality of object descriptions, each object description describing a 
corresponding one of the object files (col 1, lines 65 to col 2. lines 1-2, the 
header includes information of the type of a packet in the multimedia stream) 

- a choreography group providing information about a first group of object files (col 
1 , lines 65 to col 2, lines 1-2, packets of different types are included in the overall 
stream as a sequence of intervals wherein the type of a packet is disclosed in the 
heading are considered as a choreography group providing information about the 
object files) 

- size and type data for each object file (col 1 , lines 65 to col 2, lines 1-2, data type 
of each packet in the multimedia stream) 

- header data (col 1 , lines 65 to col 2, lines 1-2, each packet includes a header) 

- the data slices of the object files interleaved together (col 1 , lines 65 to col 2, 
lines 1-2, the overall stream is structured as a sequence of intervals called 
packets, each of which contains data of single type, indicated in a header of the 
packet itself; since data of different types are arranged in the sequence of 
intervals called packets, the packets which are equivalent to the object files, are 
interleaved together) 

- a first player pointer including an address of a player that plays the choreography 
group (col 2, lines 3-9, for each interval, the multiplexer has to decide from which 
the input stream it should take the data in order to construct the packets; this 
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implies that the multiplexer has to decide where to point to play the overall 
stream which is equivalent to the choreography group as mentioned above) 
- locating a plurality of slice size data blocks before the interleaved data slices, 
each slice size data block corresponding to one of the data slices and providing a 
size of the corresponding data slice (col 4, lines 45-53, the number of data bytes 
and the number of header bytes in each packet show the size of each packet 
which is equivalent to the data block) 
It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Caire into Robotham since Caire provides the 
choreography and the multiplexing features for a multimedia presentation. The 
combination of Caire and Robotham would provide the relationship of the objects in a 
multimedia document for effectively controlling and changing the presentation of the 
objects. 

Regarding claim 40, which is dependent on claim 31 , Robotham does not disclose a 
non-multiplex section following the multiplex section where the non-multiplex section 
includes one or more separate objects that are not played by a player as the separate 
object files are received by a receiver. Caire discloses a plurality of separate object files 
that are not played by a player as the separate object files are received by a receiver 
(col 1 . lines 37-45, ...video and audio information have to be separated again, by an 
inverse of demultiplexing process, as presentation occurs on different devices...). 
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It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Caire into Robotham since Caire provides the 
choreography and the multiplexing features for a multimedia presentation. The 
combination of Caire and Robotham would provide the relationship of the objects in a 
multimedia document for effectively controlling and changing the presentation of the 
objects. 

Claims 41-50 are for a computer system of the method claims 31-40, and are rejected 
under the same rationale. 

15. Claim 100 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Robotham et al. (US Pat No. 6.084,590, 7/4/00, filed 10/10/97, priority 4/7/97). 

Regarding independent claim 100, Robotham discloses: 

- receiving specification of media content by a user (col 7, lines 5-27, 45-50: 
representations of media objects are used by a user for producing media 
streams; col 9, lines 49-60: media elements received for producing image 
streams; col 21, lines 12-21) 

- receiving designation by the user of choreography information that indicates at 
least an intended order of presentation for the specified media content (col 7, 
lines 51-67, col 8, lines 42-48; col 9, line 10 to col 10, line 2: providing 
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choreography information for a stream of media objects ; col 21 , lines 22-32: 
selecting from the given list the objects need to be choreographed) 

- generating a single file that integrates the media content with the choreography 
information, wherein generating the single file comprises encapsulating within the 
single file at least two objects, each object including media content data and 
choreography information associated therewith, the choreography information 
comprising data defining a temporal order of presentation between the objects 
(col 7, lines 17-67: media objects are choreographed (or combined with the 
choreography information) to generate an image stream of choreographed 
objects in a virtual stage where the choreography information includes a temporal 
representation of the objects; col 8, lines 18-65: the system integrates a plurality 
of image objects with choreography information; col 21, lines 22-32; col 22, lines 
13-20) 

- before all objects of the files are received by a user, enabling the user to begin 
rendering the media content encapsulated within the file according to the 
choreography information associated with objects received (col 22, line 49 to col 
23, line 6: rendering the objects in the media file based on the choreography 
information associated with the media objects to a user; since the display of 
media segments occurs in a choreography model pixel-by-pixel basis during the 
finish rendering process, the render of media content is received as 
choreographed before all the objects of the file are received) 
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It is noted that Robotham does not specifies the author of the nnedia streann and the 
recipient of the media stream. Instead, Robotham uses the word "user" for both cases 
(col 9, lines 10-63 and col 22, line 49 to col 23, line 6). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have modified Robotham to specify such feature since in some degree, 
the author and the recipient are both the users of the media application: one uses the 
application to produce a desired media stream, and another uses the application to 
enjoy the created media stream. 

Response to Arguments 

16. Applicant's arguments with respect to claims 1-11, 13-16, 31-50, 63-66 have 
been considered but are moot in view of the new ground(s) of rejection. 

Applicants argue that Shaw does not teach or suggest beginning to render media 
content encapsulated within the file according to the choreography information 
associated with objects received before all objects of the files are received, as recited in 
claim 1 . Rather, Shaw discloses features of Microsoft PowperPoint, which is a file- 
based program that fails to teach or suggest beginning to render media content 
encapsulated within the file according to the choreography information associated with 
objects received before all objects of the files are received (Remarks, page 10). 
Examiner agrees. 
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Robotham discloses the rendering process of the media object stream where the media 
objects are choreographed (col 22, line 49 to col 23, line 6). 
Since the display of media segments occurs in a choreography model pixel-by-pixel 
basis during the finish rendering process, the render of media content is received as 
choreographed before all the objects of the file are received. 



Conclusion 

17. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 
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1 8. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Cong-Lac Huynh whose telephone number is 703-305- 
0432. The examiner can normally be reached on Mon-Fri (8:30-6:00). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Heather Herndon can be reached on 703-308-5186. The fax phone 
numbers for the organization where this application or proceeding is assigned are 703- 
746-7239 for regular communications and 707-746-7238 for After Final 
communications. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is 703-305- 
9000. 



